1. Field
The present invention generally relates to systems and methods for generating high quality plenoptic depth maps from captured images that have occluded portions of the image.
2. Related Art
In traditional photography, the area of focus of an image is configured prior to taking the picture. After the picture has been taken, the area of focus is set and the area which is out of focus cannot be made in focus. Conversely, a light-field, or a plenoptic camera, uses special lenses and sensors to capture the entire light field within the scope of a scene. Thus, a plenoptic camera is able to capture all the light traveling in every direction in every point in space. With a plenoptic camera, since the color, direction, and intensity of all light is captured, focusing is performed using software after the picture has been taken. Focusing after the picture has been taken allows the user to modify the area of the image which is in focus at any time.
In many plenoptic cameras, light enters a main (objective) lens and is then directed through an array of small microlenses to an image sensor. Each microlens may have a relatively small size, such as 100 μm, and a relatively large depth of field. This allows the camera to be focused on almost all points in a scene by capturing a series of small images from slightly different viewpoints. These viewpoints through elements of the microlens can later be extracted and manipulated by special software in order to reach a certain depth of field during post-processing. Such small handheld plenoptic cameras have now become commercially available, such as from Lytro, Inc. (Mountain View, Calif.).
Plenoptic cameras are configured to use a microlens array to capture the 4D radiance of a scene of interest. The acquired radiance, as an integral image, can be processed for either 3D scene reconstruction or synthesizing dynamic depth of field (DoF) effect. There are numerous applications for this emerging camera technology, ranging from entertainment to depth recovery for industrial and scientific applications. Some light field cameras can captures 20 different views of a scene with a 10 megapixel sensor (Adobe®, San Jose, Calif.). However, the rendered 700×700 images may have visible artifacts at occlusion boundaries. The Lytro® light field (lytro.com) camera uses an 11 megapixel sensor to acquire the radiance. However, the images generated from the camera still suffer from a low resolution of one megapixel, with some visible artifacts found around thin objects and sharp edges.